Most of the performance graphs in the offerings document have 10 years of data or less, and none of them have more than 14. I’m concerned that this analysis is overfitting.
The strategies presented in the document utilize low-fee index ETFs. Most ETFs were not around more than 10 years ago, and we decided on 2007 as a rough starting point for when to start backtests seeing as critical ETFs began to exist around that time. Our backtester also currently incorporates data from the last 15 years. That is why all of the performance graphs with the exception of one strategy have roughly 10.5 years of historical data.
I’m not sure if longer backtest periods substantially reduce the risk of overfitting, but developing and optimizing over a specific in-sample period and doing significant out-of-sample testing certainly helps, and that is part of our testing and development process. We have examined performance over longer timeframes with other tools, but exporting the data from third-party tools, merging backtest results made with mutual funds with backtest results made with ETFs, considering commissions, slippage, and unreliable data, transferring it to Excel and generating graphs, integrating it into the offerings document, and figuring out how to compare annual returns and other performance metrics with different backtest periods on the offerings document and website turned out to be quite a headache so I decided to delay displaying longer term backtests in a public facing format until we can much more easily generate graphs and performance metrics through the web app which is what we are currently working on.
There is also a segment in the FAQ titled “Antigravity Investments’ performance returns seem too good to be true, and past performance isn’t necessarily indicative of future returns. What should my expectations be for future strategy performance?” which briefly touches on strategy development.
Most of the performance graphs in the offerings document have 10 years of data or less, and none of them have more than 14. I’m concerned that this analysis is overfitting.
The strategies presented in the document utilize low-fee index ETFs. Most ETFs were not around more than 10 years ago, and we decided on 2007 as a rough starting point for when to start backtests seeing as critical ETFs began to exist around that time. Our backtester also currently incorporates data from the last 15 years. That is why all of the performance graphs with the exception of one strategy have roughly 10.5 years of historical data.
I’m not sure if longer backtest periods substantially reduce the risk of overfitting, but developing and optimizing over a specific in-sample period and doing significant out-of-sample testing certainly helps, and that is part of our testing and development process. We have examined performance over longer timeframes with other tools, but exporting the data from third-party tools, merging backtest results made with mutual funds with backtest results made with ETFs, considering commissions, slippage, and unreliable data, transferring it to Excel and generating graphs, integrating it into the offerings document, and figuring out how to compare annual returns and other performance metrics with different backtest periods on the offerings document and website turned out to be quite a headache so I decided to delay displaying longer term backtests in a public facing format until we can much more easily generate graphs and performance metrics through the web app which is what we are currently working on.
There is also a segment in the FAQ titled “Antigravity Investments’ performance returns seem too good to be true, and past performance isn’t necessarily indicative of future returns. What should my expectations be for future strategy performance?” which briefly touches on strategy development.